A Configurable Natural Language Question Benchmark for Knowledge Bases
نویسندگان
چکیده
Supporting natural language interface is crucial for accessing knowledge bases (KBs) due to the complexity of their schema. Many natural language interfaces to KBs (NLIKBs) have been built to simplify query formulation and facilitate advanced applications like personal intelligent assistant. Unfortunately, there is a lack of a comprehensive natural language question benchmark that is able to fairly evaluate different NLIKBs in terms of accuracy and runtime. In this paper, we propose a framework to construct comprehensive question benchmarks, where an array of question characteristics, including query size, commonness, modifier, answer cardinality, and paraphrasing, are made configurable. The benchmark constructed in this way delivers unprecedentedly fine-grained performance profiling, broken down by each question characteristic, which is important for performance comparison and future improvement.
منابع مشابه
An Effective Framework for Question Answering over Freebase via Reconstructing Natural Sequences
Question answering over knowledge bases has rapidly developed with the continuous expansion of resources. However, how to match the natural language question to the structured answer entities in the knowledge bases remains a major challenge. In this paper, we propose an effective framework that bridges the gap between the given question and the answer entities, by reconstructing the intermediat...
متن کاملLearning to Compose Neural Networks for Question Answering
We describe a question answering model that applies to both images and structured knowledge bases. The model uses natural language strings to automatically assemble neural networks from a collection of composable modules. Parameters for these modules are learned jointly with network-assembly parameters via reinforcement learning, with only (world, question, answer) triples as supervision. Our a...
متن کاملABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
We propose a novel attention based deep learning architecture for visual question answering task (VQA). Given an image and an image-related question, VQA returns a natural language answer. Since different questions inquire about the attributes of different image regions, generating correct answers requires the model to have questionguided attention, i.e., the attention on the regions correspond...
متن کاملTables as Semi-structured Knowledge for Question Answering
Question answering requires access to a knowledge base to check facts and reason about information. Knowledge in the form of natural language text is easy to acquire, but difficult for automated reasoning. Highly-structured knowledge bases can facilitate reasoning, but are difficult to acquire. In this paper we explore tables as a semi-structured formalism that provides a balanced compromise to...
متن کاملNatural Language Question Analysis for Querying Biomedical Linked Data
Biomedical knowledge is disseminated in knowledge bases which become increasingly available on the Web. While using biomedical Linked Data is crucial, life-science researchers may have difficulties using SPARQL language. Interfaces based on Natural Language question-answering are recognised to be suitable for querying knowledge bases. In this paper, we propose a method for translating natural l...
متن کامل